Methods and apparatuses for outputting information and calibrating camera

ABSTRACT

Embodiments of the present disclosure relate to methods and apparatuses for outputting information and calibrating a camera. The method may include: acquiring a first image, a second image, and a third image, the first image being an image photographed by a to-be-calibrated camera, the second image being a high-precision map image including a target area indicated by the first image, and the third image being a reflectance image including the target area; fusing the second image and the third image to obtain a fused image; determining a matching point pair based on points selected by a user in the first image and the fused image; and calibrating the to-be-calibrated camera based on coordinates of the matching point pair.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a divisional application of U.S. patent applicationSer. No. 16/912,360, filed on Jun. 25, 2020, which claims priority toChinese Application No. 201910907183.8, filed on Sep. 24, 2019 andentitled “Methods and Apparatuses for Outputting Information andCalibrating Camera,” the entire disclosure of which is herebyincorporated by reference.

TECHNICAL FIELD

Embodiments of the present disclosure relate to the field of computertechnology, and specifically to methods and apparatuses for outputtinginformation and calibrating a camera.

BACKGROUND

Camera calibration is one of the key research topics in the fields, suchas photogrammetry, visual detection, and computer vision, and has beenvery widely used in the fields, such as surveying and mapping,industrial control, navigation, and military. Camera calibrationprovides a quantitative relationship for correspondence and conversionbetween two-dimensional information of a visual image and a practicalthree-dimensional object world.

SUMMARY

Embodiments of the present disclosure propose methods and apparatusesfor outputting information and calibrating a camera.

In a first aspect, an embodiment of the present disclosure provides amethod for outputting information, including: displaying, in response toa first image input operation of a user, a first image selected by theuser, the first image being an image photographed by a to-be-calibratedcamera; fusing, in response to a second image input operation of theuser, a second image and a third image selected by the user, anddisplaying an obtained fused image, the second image being ahigh-precision map image including a target area indicated by the firstimage, and the third image being a reflectance image including thetarget area; displaying, in response to a point selection operation ofthe user on the first image and/or the fused image, a point selected bythe user in the first image and/or the fused image; and outputtingcoordinates of the selected point.

In some embodiments, the fusing a second image and a third imageselected by the user includes: fusing the second image and the thirdimage based on coordinates of a pixel point in the high-precision mapimage and coordinates of a pixel point in the reflectance image.

In some embodiments, the point selection operation includes a firstpoint selection operation for the first image and a second pointselection operation for the fused image; and the displaying, in responseto a point selection operation of the user on the first image and/or thefused image, a point selected by the user in the first image and/or thefused image includes: using, in response to the first point selectionoperation and the second point selection operation after the first pointselection operation, a first point indicated by the first pointselection operation and a second point indicated by the second pointselection operation as a matching point pair; and displaying thematching point pair in association on the first image and the fusedimage.

In some embodiments, the method further includes: adding, in response toan adding operation for the matching point pair, the matching point pairto a preset matching point pair set; and storing, in response to astorage operation for the matching point pair set, the matching pointpair set locally.

In some embodiments, the point selection operation includes a thirdpoint selection operation and a fourth point selection operation for thefirst image or the fused image; and the displaying, in response to apoint selection operation of the user on the first image and/or thefused image, a point selected by the user in the first image and/or thefused image includes: using, in response to the third point selectionoperation, a third point indicated by the third point selectionoperation as a starting point of an auxiliary line; using, in responseto the fourth point selection operation, a fourth point indicated by thefourth point selection operation as an ending point of the auxiliaryline; connecting, in response to a line drawing operation for thestarting point and the ending point, the starting point and the endingpoint; and displaying the starting point, the ending point, and astraight line obtained by the connecting the starting point and theending point in the first image or the fused image.

In some embodiments, the method further includes: no longer displaying,in response to a cancel operation of the user on the selected point, theselected point on which the cancel operation is performed.

In a second aspect, an embodiment of the present disclosure provides amethod for calibrating a camera, including: acquiring a first image, asecond image, and a third image, the first image being an imagephotographed by a to-be-calibrated camera, the second image being ahigh-precision map image including a target area indicated by the firstimage, and the third image being a reflectance image including thetarget area; fusing the second image and the third image to obtain afused image; determining a matching point pair based on points selectedby a user in the first image and the fused image; and calibrating theto-be-calibrated camera based on coordinates of the matching point pair.

In some embodiments, the fusing the second image and the third image toobtain a fused image includes: fusing the second image and the thirdimage based on coordinates of a pixel point in the high-precision mapimage and coordinates of a pixel point in the reflectance image.

In some embodiments, the determining a matching point pair based onpoints selected by a user in the first image and the fused imageincludes: determining a first point selected by the user in the firstimage and a second point selected in the fused image after the firstpoint is selected; and using the first point and the second point as thematching point pair.

In some embodiments, the method further includes: adding the matchingpoint pair to a preset matching point pair set; and storing the matchingpoint pair set locally.

In a third aspect, an embodiment of the present disclosure provides anapparatus for outputting information, including: a first displaying unitconfigured to display, in response to a first image input operation of auser, a first image selected by the user, the first image being an imagephotographed by a to-be-calibrated camera; a second displaying unitconfigured to fuse, in response to a second image input operation of theuser, a second image and a third image selected by the user, and displayan obtained fused image, the second image being a high-precision mapimage including a target area indicated by the first image, and thethird image being a reflectance image including the target area; a pointselecting unit configured to display, in response to a point selectionoperation of the user on the first image and/or the fused image, a pointselected by the user in the first image and/or the fused image; and anoutputting unit configured to output coordinates of the selected point.

In some embodiments, the second displaying unit is further configuredto: fuse the second image and the third image based on coordinates of apixel point in the high-precision map image and coordinates of a pixelpoint in the reflectance image.

In some embodiments, the point selection operation includes a firstpoint selection operation for the first image and a second pointselection operation for the fused image; and the second displaying unitis further configured to: use, in response to the first point selectionoperation and the second point selection operation after the first pointselection operation, a first point indicated by the first pointselection operation and a second point indicated by the second pointselection operation as a matching point pair; and display the matchingpoint pair in association on the first image and the fused image.

In some embodiments, the apparatus further includes: an adding unitconfigured to add, in response to an adding operation for the matchingpoint pair, the matching point pair to a preset matching point pair set;and a storing unit configured to store, in response to a storageoperation for the matching point pair set, the matching point pair setlocally.

In some embodiments, the point selection operation includes a thirdpoint selection operation and a fourth point selection operation for thefirst image or the fused image; and the second displaying unit isfurther configured to: use, in response to the third point selectionoperation, a third point indicated by the third point selectionoperation as a starting point of an auxiliary line; use, in response tothe fourth point selection operation, a fourth point indicated by thefourth point selection operation as an ending point of the auxiliaryline; connect, in response to a line drawing operation for the startingpoint and the ending point, the starting point and the ending point; anddisplay the starting point, the ending point, and a straight lineobtained by the connecting the starting point and the ending point inthe first image or the fused image.

In some embodiments, the apparatus further includes: a cancelling unitconfigured to no longer display, in response to a cancel operation ofthe user on the selected point, the selected point on which the canceloperation is performed.

In a fourth aspect, an embodiment of the present disclosure provides anapparatus for calibrating a camera, including: an acquiring unitconfigured to acquire a first image, a second image, and a third image,the first image being an image photographed by a to-be-calibratedcamera, the second image being a high-precision map image including atarget area indicated by the first image, and the third image being areflectance image including the target area; a fusing unit configured tofuse the second image and the third image to obtain a fused image; adetermining unit configured to determine a matching point pair based onpoints selected by a user in the first image and the fused image; and acalibrating unit configured to calibrate the to-be-calibrated camerabased on coordinates of the matching point pair.

In some embodiments, the fusing unit is further configured to: fuse thesecond image and the third image based on coordinates of a pixel pointin the high-precision map image and coordinates of a pixel point in thereflectance image.

In some embodiments, the determining unit is further configured to:determine a first point selected by the user in the first image and asecond point selected in the fused image after the first point isselected; and use the first point and the second point as the matchingpoint pair.

In some embodiments, the apparatus further includes: an adding unitconfigured to add the matching point pair to a preset matching pointpair set; and a storing unit configured to store the matching point pairset locally.

In a fifth aspect, an embodiment of the present disclosure provides anelectronic device, the electronic device including: one or moreprocessors; and a storage apparatus, storing one or more programs, wherethe one or more programs, when executed by the one or more processors,cause the one or more processors to implement any embodiment of themethod according to the first aspect.

In a sixth aspect, an embodiment of the present disclosure provides anelectronic device, the electronic device including: one or moreprocessors; and a storage apparatus, storing one or more programs, wherethe one or more programs, when executed by the one or more processors,cause the one or more processors to implement any embodiment of themethod according to the second aspect.

In a seventh aspect, an embodiment of the present disclosure provides acomputer readable medium, storing a computer program thereon, where thecomputer program, when executed by a processor, implements anyembodiment of the method according to the first aspect.

In an eighth aspect, an embodiment of the present disclosure provides acomputer readable medium, storing a computer program thereon, where thecomputer program, when executed by a processor, implements anyembodiment of the method according to the second aspect.

The methods and apparatuses for outputting information and calibrating acamera provided by embodiments of the present disclosure can support auser to perform a point selection operation on the first imagephotographed by the to-be-calibrated camera and the fused image, andoutput the coordinates of the selected point, for further cameracalibration. The methods of the present embodiment are convenient forthe user to select paired points to calibrate the camera.

BRIEF DESCRIPTION OF THE DRAWINGS

After reading detailed description of non-limiting embodiments withreference to the following accompanying drawings, other features,objectives and advantages of the present disclosure will become moreapparent.

FIG. 1 is a diagram of an example system architecture in which anembodiment of the present disclosure may be implemented;

FIG. 2 is a flowchart of a method for outputting information accordingto an embodiment of the present disclosure;

FIG. 3 is a schematic diagram of a calibration tool in the method foroutputting information according to an embodiment of the presentdisclosure;

FIG. 4 is a flowchart of a method for calibrating a camera according toan embodiment of the present disclosure;

FIG. 5 is a schematic structural diagram of an apparatus for outputtinginformation according to an embodiment of the present disclosure;

FIG. 6 is a schematic structural diagram of an apparatus for calibratinga camera according to an embodiment of the present disclosure; and

FIG. 7 is a schematic structural diagram of a computer system adapted toimplement an electronic device of embodiments of the present disclosure.

DETAILED DESCRIPTION OF EMBODIMENTS

Embodiments of present disclosure will be described below in detail withreference to the accompanying drawings. It should be appreciated thatthe specific embodiments described herein are merely used for explainingthe relevant disclosure, rather than limiting the disclosure. Inaddition, it should be noted that, for the ease of description, only theparts related to the relevant disclosure are shown in the accompanyingdrawings.

It should also be noted that some embodiments in the present disclosureand some features in the disclosure may be combined with each other on anon-conflict basis. Features of the present disclosure will be describedbelow in detail with reference to the accompanying drawings and incombination with embodiments.

FIG. 1 shows an example system architecture 100 in which a method foroutputting information, a method for calibrating a camera, an apparatusfor outputting information, or an apparatus for calibrating a camera ofembodiments of the present disclosure may be implemented.

As shown in FIG. 1, the system architecture 100 may include terminaldevices 101, 102, and 103, a network 104, and a server 105. The network104 serves as a medium providing a communication link between theterminal devices 101, 102, and 103, and the server 105. The network 104may include various types of connections, such as wired or wirelesscommunication links, or optical cables.

A user may interact with the server 105 using the terminal devices 101,102, and 103 via the network 104, e.g., to receive or send a message.The terminal devices 101, 102, and 103 may be provided with variouscommunication client applications, such as an image browsingapplication, an image processing application, a web browser application,an instant messaging tool, an email client, and social platformsoftware.

The terminal devices 101, 102, and 103 may be hardware, or may besoftware. When the terminal devices 101, 102, and 103 are hardware, theterminal devices may be various electronic devices having displayscreens and supporting image processing, including but not limited tosmart phones, tablet computers, laptop portable computers, desktopcomputers, and the like. When the terminal devices 101, 102, and 103 aresoftware, the terminal devices may be installed in the above-listedelectronic devices, or may be implemented as a plurality of softwareprograms or software modules (e.g., software programs or softwaremodules for providing distributed services), or may be implemented as asingle software program or software module. This is not specificallylimited here.

The server 105 may be a server providing various services, such as aback-end image server providing support for images displayed on theterminal devices 101, 102, and 103. The back-end image server canprocess, e.g., analyze, received data, such as an image.

It should be noted that the server 105 may be hardware, or may besoftware. When the server 105 is hardware, the server may be implementedas a distributed server cluster composed of a plurality of servers, ormay be implemented as a single server. When the server 105 is software,the server may be implemented as a plurality of software programs orsoftware modules (e.g., software programs or software modules forproviding distributed services), or may be implemented as a singlesoftware program or software module. This is not specifically limitedhere.

It should be noted that the method for outputting information providedby some embodiments of the present disclosure is generally executed bythe terminal devices 101, 102, and 103; and the method for calibrating acamera may be executed by the terminal devices 101, 102, and 103, or maybe executed by the server 105. Accordingly, the apparatus for outputtinginformation is generally provided in the terminal devices 101, 102, and103; and the apparatus for calibrating a camera may be provided in theterminal devices 101, 102, and 103, or may be provided in the server105.

It should be understood that the numbers of terminal devices, networks,and servers in FIG. 1 are merely illustrative. Any number of terminaldevices, networks, and servers may be provided based on actualrequirements.

With further reference to FIG. 2, a process 200 of a method foroutputting information according to an embodiment of the presentdisclosure is shown. The method for outputting information includesfollowing steps.

Step 201: displaying, in response to a first image input operation of auser, a first image selected by the user.

In the present embodiment, an executing body (e.g., the terminal devices101, 102, and 103 shown in FIG. 1) of the method for outputtinginformation may detect an operation of the user. If the first imageinput operation of the user is detected, then the first image selectedby the user can be displayed. Specifically, a calibration tool may beinstalled in the executing body, and the calibration tool may include afirst image input button. The user can perform the first image inputoperation by clicking the first image input button or using a presetshortcut key. The executing body can display an image input window afterdetecting the first image input operation, such that the user selectsthe first image to be inputted into the calibration tool. Thecalibration tool may further include a display window, and the executingbody can display the first image in the display window.

In the present embodiment, the first image is an image photographed by ato-be-calibrated camera. The to-be-calibrated camera may be fixed at acertain position of a road or an intersection, and is used forphotographing a road surface image.

Step 202: fusing, in response to a second image input operation of theuser, a second image and a third image selected by the user, anddisplaying an obtained fused image.

The executing body can further detect the second image input operationof the user, fuse the second image and the third image selected by theuser upon detecting the second image input operation, to obtain thefused image, and display the fused image. Specifically, the calibrationtool may include a second image input button. The user can perform thesecond image input operation by clicking the second image input buttonor using a preset shortcut key. The executing body can display the imageinput window after detecting the second image input operation, such thatthe user selects the to-be-fused second image and third image.

In the present embodiment, the second image is a high-precision mapimage including a target area indicated by the first image, and thethird image is a reflectance image including the target area. Here, thehigh-precision map has more abundant map information than ordinary maps,such as lane lines, lane boundaries, zebra crossings, stop lines,traffic lights, and traffic signs, as well as height, slope distance,and other information. The reflectance image refers to an image obtainedby projecting point cloud data to the ground. The point cloud datainclude building information, road information, and the like. It can beunderstood that both the high-precision map image and the reflectanceimage have coordinate information.

The executing body may fuse the high-precision map image and thereflectance image based on coordinate information therein, or may fusethe high-precision map image and the reflectance image based on roadinformation therein. Here, the fusing the high-precision map image andthe reflectance image may refer to image processing on thehigh-precision map image and the reflectance image, such that theobtained fused image displays information included in both thehigh-precision map image and the reflectance image. Thus, the user canconveniently select a point in the fused image.

In some alternative implementations of the present embodiment, theexecuting body may fuse the second image and the third image through thefollowing step that is not shown in FIG. 2: fusing the second image andthe third image based on coordinates of a pixel point in thehigh-precision map image and coordinates of a pixel point in thereflectance image.

In the present implementation, the executing body may determine, basedon the coordinates of the pixel point in the high-precision map image,the pixel point in the reflectance image having the same coordinates asthe coordinates of the pixel point in the high-precision map image. Thefused image can be obtained by aligning the pixel points of the samecoordinates.

In some alternative implementations of the present embodiment, a windowdisplaying the first image and a window displaying the fused image maybe arranged left and right, or may be arranged up and down. Thus, theuser can compare the first image and the fused image more conveniently,thereby conveniently selecting paired points.

Step 203: displaying, in response to a point selection operation of theuser on the first image and/or the fused image, a point selected by theuser in the first image and/or the fused image.

The user may perform the point selection operation on the displayedfirst image and/or the fused image. Specifically, the user may performthe point selection operation by clicking a button in the calibrationtool or using a shortcut key. The user may select a point only in thefirst image or the fused image, or may select a point in the firstimage, and then select a point in the fused image. In the presentembodiment, the executing body may display the point selected by theuser in the first image and/or the fused image. It can be understoodthat if the user selects a point in the first image, then the selectedpoint can be displayed only in the first image. Similarly, if the userselects a point only in the fused image, then the selected point can bedisplayed only in the fused image.

Step 204: outputting coordinates of the selected point.

On completion of the point selection operation, the executing body canoutput the coordinates of the selected point. Specifically, for thepoint selected by the user in the first image, the executing body canoutput coordinates of the selected point in the first image. For thepoint selected by the user in the fused image, the executing body canoutput projection coordinates of the selected point in the fused image,or can output three-dimensional coordinates of the selected point. Thatis, for the point selected in the fused image, the executing body may ormay not output height information of the point.

In the present embodiment, the executing body may output the coordinatesof the selected point to a server, such that the server calibrates theto-be-calibrated camera. Alternatively, the coordinates of the selectedpoint may also be outputted to a calibration algorithm running locally,to calibrate the to-be-calibrated camera.

The method for outputting information provided by embodiments of thepresent disclosure may allow the user to more conveniently select thepaired points, thus facilitating calibrating the camera.

In some alternative implementations of the present embodiment, the pointselection operation includes a first point selection operation for thefirst image and a second point selection operation for the fused image.The above step 203 may specifically include the following steps that arenot shown in FIG. 2: using, in response to the first point selectionoperation and a second point selection operation after the first pointselection operation, a first point indicated by the first pointselection operation and a second point indicated by the second pointselection operation as a matching point pair; and displaying thematching point pair in association on the first image and the fusedimage.

If the executing body detects the first point selection operation, anddetects the second point selection after the first point selectionoperation, then the executing body may use the first point indicated bythe first point selection operation and the second point indicated bythe second point selection operation as the matching point pair. Thatis, if the user completes selecting the first point on the first image,and then continues to complete selecting the second point on the fusedimage, then the executing body may use the first point and the secondpoint as the matching point pair. The executing body can further displaythe matching point pair in association on the first image and the fusedimage. Here, the displaying the matching point pair in association mayrefer to displaying the matching point pair in the same color, or mayrefer to displaying the matching point pair using the same identifier.

In some alternative implementations of the present embodiment, themethod may further include the following steps that not shown in FIG. 2:adding, in response to an adding operation for the matching point pair,the matching point pair to a preset matching point pair set; andstoring, in response to a storage operation for the matching point pairset, the matching point pair set locally.

In the present implementation, the user can further perform the addingoperation on the matching point pair. The user can perform the addingoperation via an adding button in the calibration tool or a shortcutkey. After detecting the adding operation, the executing body may addthe matching point pair to the preset matching point pair set. It can beunderstood that, when the number of matching point pairs in the matchingpoint pair set is enough, the executing body can calibrate theto-be-calibrated camera using each matching point pair.

The executing body can further detect whether the user performs astorage operation for the matching point pair set. The user can performthe storage operation via a storage button in the calibration tool or ashortcut key. After detecting the storage operation, the executing bodymay store the matching point pair set locally.

In some alternative implementations of the present embodiment, the pointselection operation may further include a third point selectionoperation and a fourth point selection operation for the first image orthe fused image. Then, the executing body may also implement the step203 through the following steps that are not shown in FIG. 2: using, inresponse to the third point selection operation, a third point indicatedby the third point selection operation as a starting point of anauxiliary line; using, in response to the fourth point selectionoperation, a fourth point indicated by the fourth point selectionoperation as an ending point of the auxiliary line; connecting, inresponse to a line drawing operation for the starting point and theending point, the starting point and the ending point; and displayingthe starting point, the ending point, and a straight line obtained bythe connecting the starting point and the ending point in the firstimage or the fused image.

In the present implementation, the user may further select two points inthe first image or the fused image, and connect the two selected points,to obtain the straight line. In practical application, there may beillegibleness in the first image or the fused image, and the user maydetermine a point to be practically selected by drawing an auxiliaryline. Specifically, the executing body may use the third point indicatedby the third point selection operation as the starting point of theauxiliary line, and use the fourth point indicated by the fourth pointselection operation as the ending point of the auxiliary line; connectthe starting point and the ending point after detecting the line drawingoperation for the starting point and the ending point; and display thestraight line obtained by the connecting the starting point and theending point in the first image or the fused image.

In some alternative implementations of the present embodiment, themethod may further include the following step that is not shown in FIG.2: no longer displaying, in response to a cancel operation of the useron the selected point, the selected point on which the cancel operationis performed.

In the present implementation, after the user selects a point, if theuser would like to cancel the selected point, then the user can performa cancel (delete) operation. After the executing body detects the canceloperation, the selected point on which the cancel operation is performedcan be no longer displayed.

In the present embodiment, the user can further perform a zoom operationon the displayed first image and/or fused image, to facilitate selectingthe point in the first image and/or the fused image.

Referring to FIG. 3, a schematic diagram of a user interface of acalibration tool in the method for outputting information according toan embodiment of the present disclosure is shown. By using buttons onthe interface, a user can perform various operations to select points ina first image (left) and a fused image (right), respectively.

With further reference to FIG. 4, a process 400 of a method forcalibrating a camera according to an embodiment of the presentdisclosure is shown. As shown in FIG. 4, the method for calibrating acamera according to the present embodiment may include following steps.

Step 401: acquiring a first image, a second image, and a third image.

In the present embodiment, an executing body (e.g., the terminal devices101, 102, and 103, or the server 105 shown in FIG. 1) of the method forcalibrating a camera may acquire the first image, the second image, andthe third image. When the executing body is a terminal device, theexecuting body may acquire the first image, the second image, and thethird image locally based on an operation of a user. For example, acalibration tool may be installed in the terminal device, and then theuser may select the first image, the second image, and the third imagelocally by clicking a button in the calibration tool or using a shortcutkey. When the executing body is a server, the executing body may acquirethe first image, the second image, and the third image from otherdevices.

The first image is an image photographed by a to-be-calibrated camera.The to-be-calibrated camera may be fixed at a certain position of a roador an intersection, and is used for photographing a road surface image.The second image is a high-precision map image including a target areaindicated by the first image. The third image is a reflectance imageincluding the target area. Here, the high-precision map has moreabundant map information than ordinary maps, such as lane lines, laneboundaries, zebra crossings, stop lines, traffic lights, and trafficsigns, as well as height, slope distance, and other information. Thereflectance image refers to an image obtained by projecting point clouddata to the ground. The point cloud data include building information,road information, and the like. It can be understood that both thehigh-precision map image and the reflectance image have coordinateinformation.

Step 402: fusing the second image and the third image to obtain a fusedimage.

The executing body may fuse the high-precision map image and thereflectance image based on coordinate information therein, or may fusethe high-precision map image and the reflectance image based on roadinformation therein. Here, the fusing the high-precision map image andthe reflectance image may refer to aligning the high-precision map imageand the reflectance image, and the obtained fused image displaysinformation of the high-precision map image and the reflectance image.Thus, the user can conveniently select a point in the fused image.

In some alternative implementations of the present embodiment, theexecuting body may fuse the second image and the third image through thefollowing step that is not shown in FIG. 4: fusing the second image andthe third image based on coordinates of a pixel point in thehigh-precision map image and coordinates of a pixel point in thereflectance image.

In the present implementation, the executing body may determine, basedon the coordinates of the pixel point in the high-precision map image,the pixel point in the reflectance image having the same coordinates asthe coordinates of the pixel point in the high-precision map image. Thefused image can be obtained by superposing the pixel points of the samecoordinates.

Step 403: determining a matching point pair based on points selected bya user in the first image and the fused image.

In the present embodiment, the user may select a point in the firstimage and a point in the fused image respectively, and the executingbody may use the points selected by the user as a matching point pair.Specifically, when the executing body is a terminal device, theexecuting body may display the first image and the fused image on ascreen, and the user may select the points in the first image and thefused image respectively through an input device of the terminal device.When the executing body is a server, the executing body may send thefirst image and the fused image to a terminal device used by the user.After completing selecting the points, the user sends the first imageand the fused image including the selected points to the server.

In some alternative implementations of the present embodiment, theexecuting body may implement the above step 403 through the followingsteps that are not shown in FIG. 4: determining a first point selectedby the user in the first image and a second point selected in the fusedimage after the first point is selected; and using the first point andthe second point as a matching point pair.

If the executing body detects the user selecting the first point in thefirst image, and then detects the user selecting the second point in thefused image, then the executing body may use the first point and thesecond point as the matching point pair. That is, if the user completesselecting the first point in the first image, and then continues tocomplete selecting the second point in the fused image, then theexecuting body may use the first point and the second point as thematching point pair.

Step 404: calibrating a to-be-calibrated camera based on coordinates ofthe matching point pair.

In the present embodiment, after determining the matching point pair,the executing body may first determine the coordinates of the matchingpoint pair. For the point selected by the user in the first image, theexecuting body can output coordinates of the selected point in the firstimage. For the point selected by the user in the fused image, theexecuting body can output projection coordinates of the selected pointin the fused image, or can output three-dimensional coordinates of theselected point. That is, for the point selected in the fused image, theexecuting body may or may not output height information of the point.

In the present embodiment, coordinates of points in the fused image arecoordinates in a geodetic coordinate system, and coordinates of pointsin the first image are coordinates in a camera coordinate system.Through an existing calibration algorithm, calibration parameters of thecamera coordinate system and the geodetic coordinate system can beobtained, thereby realizing the calibration of the to-be-calibratedcamera.

In some alternative implementations of the present embodiment, themethod may further include the following steps that not shown in FIG. 4:adding the matching point pair to a preset matching point pair set; andstoring the matching point pair set locally.

In the present implementation, the executing body may further add thematching point pair to the preset matching point pair set. The matchingpoint pair set may be further stored in the executing body locally.Thus, when the number of matching point pairs in the matching point pairset is enough, the executing body can calibrate the to-be-calibratedcamera using coordinates of each matching point pair.

The method for calibrating a camera provided by embodiments of thepresent disclosure can support the user to select the matching pointpair using an image collected by the to-be-calibrated camera, and ahigh-precision map image and a reflectance image of the same area, andcalibrates the camera using the matching point pair, thereby improvingthe calibration efficiency of manual camera calibration.

With further reference to FIG. 5, as an implementation of the methodshown in the above FIG. 2, an embodiment of the present disclosureprovides an apparatus for outputting information. An embodiment of theapparatus may correspond to an embodiment of the method shown in FIG. 2.The apparatus may be specifically applied to various electronic devices.

As shown in FIG. 5, the apparatus 500 for outputting information of thepresent embodiment includes: a first displaying unit 501, a seconddisplaying unit 502, a point selecting unit 503, and an outputting unit504.

The first displaying unit 501 is configured to display, in response to afirst image input operation of a user, a first image selected by theuser. The first image is an image photographed by a to-be-calibratedcamera.

The second displaying unit 502 is configured to fuse, in response to asecond image input operation of the user, a second image and a thirdimage selected by the user, and display an obtained fused image. Thesecond image is a high-precision map image including a target areaindicated by the first image, and the third image is a reflectance imageincluding the target area.

The point selecting unit 503 is configured to display, in response to apoint selection operation of the user on the first image and/or thefused image, a point selected by the user in the first image and/or thefused image.

The outputting unit 504 is configured to output coordinates of theselected point.

In some alternative implementations of the present embodiment, thesecond displaying unit 502 may be further configured to: fuse the secondimage and the third image based on coordinates of a pixel point in thehigh-precision map image and coordinates of a pixel point in thereflectance image.

In some alternative implementations of the present embodiment, the pointselection operation includes a first point selection operation for thefirst image and a second point selection operation for the fused image.The second displaying unit 502 may be further configured to: use, inresponse to the first point selection operation and the second pointselection operation after the first point selection operation, a firstpoint indicated by the first point selection operation and a secondpoint indicated by the second point selection operation as a matchingpoint pair; and display the matching point pair in association on thefirst image and the fused image.

In some alternative implementations of the present embodiment, theapparatus 500 may further include an adding unit and a storing unit thatare not shown in FIG. 5.

The adding unit is configured to add, in response to an adding operationfor the matching point pair, the matching point pair to a presetmatching point pair set.

The storing unit is configured to store, in response to a storageoperation for the matching point pair set, the matching point pair setlocally.

In some alternative implementations of the present embodiment, the pointselection operation includes a third point selection operation and afourth point selection operation for the first image or the fused image.The second displaying unit 502 may be further configured to: use, inresponse to the third point selection operation, a third point indicatedby the third point selection operation as a starting point of anauxiliary line; use, in response to the fourth point selectionoperation, a fourth point indicated by the fourth point selectionoperation as an ending point of the auxiliary line; connect, in responseto a line drawing operation for the starting point and the ending point,the starting point and the ending point; and display the starting point,the ending point, and a straight line obtained by the connecting thestarting point and the ending point in the first image or the fusedimage.

In some alternative implementations of the present embodiment, theapparatus 500 may further include a cancelling unit that is not shown inFIG. 5, and is configured to no longer display, in response to a canceloperation of the user on the selected point, the selected point on whichthe cancel operation is performed.

It should be understood that the unit 501 to unit 504 in the apparatus500 for outputting information correspond to the steps in the methoddescribed in FIG. 2 respectively. Therefore, the operations and featuresdescribed above for the method for outputting information also apply tothe apparatus 500 and the units included therein. The description willnot be repeated here.

With further reference to FIG. 6, as an implementation of the methodshown in the above FIG. 3, an embodiment of the present disclosureprovides an apparatus for calibrating a camera. An embodiment of theapparatus may correspond to an embodiment of the method shown in FIG. 4.The apparatus may be specifically applied to various electronic devices.

As shown in FIG. 6, the apparatus 600 for calibrating a camera of thepresent embodiment includes: an acquiring unit 601, a fusing unit 602, adetermining unit 603, and a calibrating unit 604.

The acquiring unit 601 is configured to acquire a first image, a secondimage, and a third image. The first image is an image photographed by ato-be-calibrated camera, the second image is a high-precision map imageincluding a target area indicated by the first image, and the thirdimage is a reflectance image including the target area.

The fusing unit 602 is configured to fuse the second image and the thirdimage to obtain a fused image.

The determining unit 603 is configured to determine a matching pointpair based on points selected by a user in the first image and the fusedimage.

The calibrating unit 604 is configured to calibrate the to-be-calibratedcamera based on coordinates of the matching point pair.

In some alternative implementations of the present embodiment, thefusing unit 602 may be further configured to: fuse the second image andthe third image based on coordinates of a pixel point in thehigh-precision map image and coordinates of a pixel point in thereflectance image.

In some alternative implementations of the present embodiment, thedetermining unit 603 may be further configured to: determine a firstpoint selected by the user in the first image and a second pointselected in the fused image after the first point is selected; and usethe first point and the second point as a matching point pair.

In some alternative implementations of the present embodiment, theapparatus 600 may further include an adding unit and a storing unit thatare not shown in FIG. 6.

The adding unit is configured to add the matching point pair to a presetmatching point pair set.

The storing unit is configured to store the matching point pair setlocally.

It should be understood that the unit 601 to unit 604 in the apparatus600 for calibrating a camera correspond to the steps in the methoddescribed in FIG. 4 respectively. Therefore, the operations and featuresdescribed above for the method for calibrating a camera also apply tothe apparatus 600 and the units included therein. The description willnot be repeated here.

Referring to FIG. 7 below, a schematic structural diagram adapted toimplement an electronic device (e.g., the server or the terminal devicein FIG. 1) 700 of embodiments of the present disclosure is shown. Theterminal device in some embodiments of the present disclosure mayinclude, but is not limited to, mobile terminals such as a mobile phone,a notebook computer, and a vehicle terminal (e.g., a vehicle navigationterminal), and fixed terminals such as a desktop computer. Theelectronic device shown in FIG. 7 is merely an example, and should notlimit the functions and scope of use of some embodiments of the presentdisclosure.

As shown in FIG. 7, the electronic device 700 may include a processingapparatus (e.g., a central processing unit, or a graphics processor)701, which may execute various appropriate actions and processes inaccordance with a program stored in a read-only memory (ROM) 702 or aprogram loaded into a random access memory (RAM) 703 from a storageapparatus 708. The RAM 703 further stores various programs and datarequired by operations of the electronic device 700. The processingapparatus 701, the ROM 702, and the RAM 703 are connected to each otherthrough a bus 704. An input/output (I/O) interface 705 is also connectedto the bus 704.

In general, the following apparatuses may be connected to the I/Ointerface 705: an input apparatus 706 including a touch screen, a touchpad, a keyboard, a mouse, a camera, a microphone, an accelerometer, agyroscope, or the like; an output apparatus 707 including a liquidcrystal display device (LCD), a speaker, a vibrator, or the like; astorage apparatus 708 including a magnetic tape, a hard disk, or thelike; and a communication apparatus 709. The communication apparatus 709may allow the electronic device 700 to exchange data with other devicesthrough wireless or wired communication. While FIG. 7 shows theelectronic device 700 having various apparatuses, it should beunderstood that it is not necessary to implement or provide all of theapparatuses shown in the figure. More or fewer apparatuses may bealternatively implemented or provided. Each block shown in FIG. 7 mayrepresent an apparatus, or represent a plurality of apparatuses asrequired.

In particular, according to some embodiments of the present disclosure,the process described above with reference to the flow chart may beimplemented in a computer software program. For example, someembodiments of the present disclosure include a computer programproduct, which includes a computer program that is tangibly embedded ina computer readable medium. The computer program includes program codesfor executing the method as shown in the flow chart. In such anembodiment, the computer program may be downloaded and installed from anetwork via the communication apparatus 709, or be installed from thestorage apparatus 708, or be installed from the ROM 702. The computerprogram, when executed by the processing apparatus 701, implements theabove functions as defined by the method of some embodiments of thepresent disclosure. It should be noted that the computer readable mediumaccording to some embodiments of the present disclosure may be acomputer readable signal medium or a computer readable medium or anycombination of the above two. An example of the computer readable mediummay include, but is not limited to: electric, magnetic, optical,electromagnetic, infrared, or semiconductor systems, apparatuses,elements, or a combination of any of the above. A more specific exampleof the computer readable medium may include, but is not limited to:electrical connection with one or more pieces of wire, a portablecomputer disk, a hard disk, a random access memory (RAM), a read onlymemory (ROM), an erasable programmable read only memory (EPROM or flashmemory), an optical fiber, a portable compact disk read only memory(CD-ROM), an optical memory, a magnetic memory, or any suitablecombination of the above. In some embodiments of the present disclosure,the computer readable medium may be any tangible medium containing orstoring programs, which may be used by, or used in combination with, acommand execution system, apparatus or element. In some embodiments ofthe present disclosure, the computer readable signal medium may includea data signal in the base band or propagating as a part of a carrierwave, in which computer readable program codes are carried. Thepropagating data signal may take various forms, including but notlimited to an electromagnetic signal, an optical signal, or any suitablecombination of the above. The computer readable signal medium may alsobe any computer readable medium except for the computer readable medium.The computer readable medium is capable of transmitting, propagating ortransferring programs for use by, or used in combination with, a commandexecution system, apparatus or element. The program codes contained onthe computer readable medium may be transmitted with any suitablemedium, including but not limited to: wireless, wired, optical cable, RFmedium, etc., or any suitable combination of the above.

The computer readable medium may be included in the above electronicdevice; or a stand-alone computer readable medium without beingassembled into the electronic device. The computer readable mediumstores one or more programs. The one or more programs, when executed bythe electronic device, cause the electronic device to: display, inresponse to a first image input operation of a user, a first imageselected by the user in a calibration tool, the first image being animage photographed by a to-be-calibrated camera; fuse, in response to asecond image input operation of the user, a second image and a thirdimage selected by the user, and display an obtained fused image in thecalibration tool, the second image being a high-precision map imageincluding a target area indicated by the first image, and the thirdimage being a reflectance image including the target area; display, inresponse to a point selection operation of the user on the first imageand/or the fused image, a point selected by the user in the first imageand/or the fused image; and output coordinates of the selected point.Alternatively, the one or more programs, when executed by the electronicdevice, cause the electronic device to: acquire a first image, a secondimage, and a third image, the first image being an image photographed bya to-be-calibrated camera, the second image being a high-precision mapimage including a target area indicated by the first image, and thethird image being a reflectance image including the target area; fusethe second image and the third image to obtain a fused image; determinea matching point pair based on points selected by a user in the firstimage and the fused image; and calibrate the to-be-calibrated camerabased on coordinates of the matching point pair.

A computer program code for executing operations in some embodiments ofthe present disclosure may be compiled using one or more programminglanguages or combinations thereof. The programming languages includeobject-oriented programming languages, such as Java, Smalltalk or C++,and also include conventional procedural programming languages, such as“C” language or similar programming languages. The program code may becompletely executed on a user's computer, partially executed on a user'scomputer, executed as a separate software package, partially executed ona user's computer and partially executed on a remote computer, orcompletely executed on a remote computer or server. In a circumstanceinvolving a remote computer, the remote computer may be connected to auser's computer through any network, including local area network (LAN)or wide area network (WAN), or be connected to an external computer (forexample, connected through the Internet using an Internet serviceprovider).

The flow charts and block diagrams in the accompanying drawingsillustrate architectures, functions and operations that may beimplemented according to the systems, methods and computer programproducts of the various embodiments of the present disclosure. In thisregard, each of the blocks in the flow charts or block diagrams mayrepresent a module, a program segment, or a code portion, said module,program segment, or code portion including one or more executableinstructions for implementing specified logical functions. It should befurther noted that, in some alternative implementations, the functionsdenoted by the blocks may also occur in a sequence different from thesequences shown in the figures. For example, any two blocks presented insuccession may be executed substantially in parallel, or they maysometimes be executed in a reverse sequence, depending on the functionsinvolved. It should be further noted that each block in the blockdiagrams and/or flow charts as well as a combination of blocks in theblock diagrams and/or flow charts may be implemented using a dedicatedhardware-based system executing specified functions or operations, or bya combination of dedicated hardware and computer instructions.

The units involved in some embodiments of the present disclosure may beimplemented by software, or may be implemented by hardware. Thedescribed units may also be provided in a processor, for example,described as: a processor including a first displaying unit, a seconddisplaying unit, a point selecting unit, and an outputting unit.Alternatively, a processor includes an acquiring unit, a fusing unit, adetermining unit, and a calibrating unit. The names of the units do notconstitute a limitation to such units themselves in some cases. Forexample, the acquiring unit may be further described as “a unitconfigured to acquire a first image, a second image, and a third image.”

The above description only provides an explanation of embodiments of thepresent disclosure and the technical principles used. It should beappreciated by those skilled in the art that the inventive scope of thepresent disclosure is not limited to the technical solutions formed bythe particular combinations of the above-described technical features.The inventive scope should also cover other technical solutions formedby any combinations of the above-described technical features orequivalent features thereof without departing from the concept of thepresent disclosure. Technical schemes formed by the above-describedfeatures being interchanged with, but not limited to, technical featureswith similar functions disclosed in the present disclosure are examples.

What is claimed is:
 1. A method for calibrating a camera, comprising:acquiring a first image, a second image, and a third image, the firstimage being an image photographed by a to-be-calibrated camera, thesecond image being a high-precision map image comprising a target areaindicated by the first image, and the third image being a reflectanceimage comprising the target area; fusing the second image and the thirdimage to obtain a fused image; determining a matching point pair basedon points selected by a user in the first image and the fused image; andcalibrating the to-be-calibrated camera based on coordinates of thematching point pair.
 2. The method according to claim 1, wherein thefusing the second image and the third image to obtain a fused imagecomprises: fusing the second image and the third image based oncoordinates of a pixel point in the high-precision map image andcoordinates of a pixel point in the reflectance image.
 3. The methodaccording to claim 1, wherein the determining a matching point pairbased on points selected by a user in the first image and the fusedimage comprises: determining a first point selected by the user in thefirst image and a second point selected in the fused image after thefirst point is selected; and using the first point and the second pointas the matching point pair.
 4. The method according to claim 3, whereinthe method further comprises: adding the matching point pair to a presetmatching point pair set; and storing the matching point pair setlocally.
 5. An apparatus for calibrating a camera, comprising: at leastone processor; and a memory storing instructions, the instructions whenexecuted by the at least one processor, causing the at least oneprocessor to perform operations, the operations comprising: acquiring afirst image, a second image, and a third image, the first image being animage photographed by a to-be-calibrated camera, the second image beinga high-precision map image comprising a target area indicated by thefirst image, and the third image being a reflectance image comprisingthe target area; fusing the second image and the third image to obtain afused image; determining a matching point pair based on points selectedby a user in the first image and the fused image; and calibrating theto-be-calibrated camera based on coordinates of the matching point pair.6. The apparatus according to claim 5, wherein the fusing the secondimage and the third image to obtain a fused image comprises: fusing thesecond image and the third image based on coordinates of a pixel pointin the high-precision map image and coordinates of a pixel point in thereflectance image.
 7. The apparatus according to claim 5, wherein thedetermining a matching point pair based on points selected by a user inthe first image and the fused image comprises: determining a first pointselected by the user in the first image and a second point selected inthe fused image after the first point is selected; and using the firstpoint and the second point as the matching point pair.
 8. The apparatusaccording to claim 7, wherein the operations further comprise: addingthe matching point pair to a preset matching point pair set; and storingthe matching point pair set locally.
 9. A non-transitory computerreadable medium, storing a computer program thereon, wherein thecomputer program, when executed by a processor, causes the processor toimplement operations, the operations comprising: acquiring a firstimage, a second image, and a third image, the first image being an imagephotographed by a to-be-calibrated camera, the second image being ahigh-precision map image comprising a target area indicated by the firstimage, and the third image being a reflectance image comprising thetarget area; fusing the second image and the third image to obtain afused image; determining a matching point pair based on points selectedby a user in the first image and the fused image; and calibrating theto-be-calibrated camera based on coordinates of the matching point pair.10. The computer readable medium according to claim 9, wherein thefusing the second image and the third image to obtain a fused imagecomprises: fusing the second image and the third image based oncoordinates of a pixel point in the high-precision map image andcoordinates of a pixel point in the reflectance image.
 11. The computerreadable medium according to claim 9, wherein the determining a matchingpoint pair based on points selected by a user in the first image and thefused image comprises: determining a first point selected by the user inthe first image and a second point selected in the fused image after thefirst point is selected; and using the first point and the second pointas the matching point pair.
 12. The computer readable medium accordingto claim 11, wherein the operations further comprise: adding thematching point pair to a preset matching point pair set; and storing thematching point pair set locally.